An image pickup element (image sensor) used, for example, in an image pickup apparatus has a structure in which a color filter that transmits a particular wavelength component light (R, G, and B) is affixed onto an element front surface in unit of respective pixels. For example, a filter having an RGB pattern illustrated in FIG. 1(a) is utilized. In a color image generation processing using an output signal of the image pickup element, a processing of reconstructing necessary color components by a set of a plurality of pixels is performed. Various types exists for color patterns of the color filter, but Bayer pattern illustrated in FIG. 1(a) constructed by three types of filters that transmit only a particular wavelength light of red (R), green (G), and blue (B) is often utilized.
In recent years, a miniaturization of the pixel in the image pickup element (image sensor) has advanced, and along with this, a problem has occurred that the amount of incident light on the respective pixels is reduced, and an S/N ratio deteriorates. To solve this problem, as illustrated in FIG. 1(b), an image sensor (image pickup element) further having white (W: White) that widely transmits light in a visible light range in addition to the filters that transmit only the particular wavelength light such as RGB is proposed. FIG. 1(b) illustrates an example of a filter having an RGBW pattern. A W pixel in the RGBW pattern illustrated in FIG. 1(b) is a filter that widely transmits the light in the visible light range.
Such an image pickup element mounted with the color filter having the white (W: White) pixel is disclosed, for example, in Patent Literature 1 (US Patent No. 2007/0024879) and Patent Literature 2 (US Patent No. 2007/0024934).
By using the image pickup element (image sensor) having the color filter having the white (W: White) pixel illustrated in FIG. 1(b), a transmission light factor of the filter is increased, and a higher sensitivity can be realized.
However, as a problematic point for the RGBW-type element, the following problematic point exists.
Either the RGB pattern illustrated in FIG. 1(a) or the RGBW pattern illustrated in FIG. 1(b) is a one sheet of the element where filters of any of R, G, B, and W are arrayed in a mosaic manner, that is, a single image sensor. Therefore, when a color image is generated, a demosaic processing serving as a color coding for generating RGB pixel values corresponding to the respective pixels is necessary.
In the RGBW pattern illustrated in FIG. 1(b), sampling rates for R, G, and B components are decreased as compared with the RGB pattern illustrated in FIG. 1(a). As a result, at the time of a color image generation processing, in a case where obtained data of the element with the RGBW pattern illustrated in FIG. 1(b) is used, as compared with the RGB pattern illustrated in FIG. 1(a), a problem occurs that a false color tends to be generated. Also, as a wavelength component of white (W) includes all the wavelength components of R, G, and B, when an optical lens with a large chromatic aberration is used, light collecting rates for the RGB wavelength components drop as compared with one with a single color component, and a problem also occurs that a decrease in resolution is invited. This problematic point becomes further conspicuous as the miniaturization of the pixel advances.
As a prevention technique for the resolution degrease due to the chromatic aberration of the optical lens, it is effective to suppress the generation of the chromatic aberration by combining lenses different in refractive index, but in this case, a new problem occurs that costs are increased as the number of optical lenses is increased. Furthermore, with such a configuration, a problem also occurs that the false color problem due to the decrease in sampling rates for the above-mentioned RGB components becomes more conspicuous.
Also, as the respective pixels of the single image sensor has only information on a signal color component, a demosaic processing of obtaining RGB pixel values corresponding to all the pixels is performed to obtain a color image from discretely obtained R, G, B, and W signals. At the time of this demosaic processing, an interpolation processing is performed while it is assumed that the color ratio is kept substantially constant and a strong color correlation exists in a local area. To be specific, when a pixel value of a particular color of a certain pixel is calculated, an interpolation method using surrounding pixel values is widely employed. With regard to this method, for example, a description is made in Patent Literature 3 (Japanese Unexamined Patent Application Publication No. 2009-17544). However, in the vicinity of an edge, the above-mentioned assumption, that is, the color ratio is kept substantially constant and the strong color correlation exists in the local area, is not established. As a result, a problem occurs that a false color tends to be generated in the vicinity of the edge.